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EXECUTIVE  SUMMARY 


The  Ordnance  Detection  Expert  Support  Application  (ODESA)  is  a  prototype  data  fusion 
system  that  utilizes  genetic  algorithms  and  a  heuristic  rule  base  to  provide  a  more  accvirate  and 
reliable  accounting  of  the  location  and  identification  of  buried  unexploded  ordnance  (UXO). 
ODESA  is  a  component  of  the  Unexploded  Ordnance  Clearance  Technology  Program  (UXO-CTP) 
sponsored  by  the  U.  S.  Army  Environmental  Center  and  managed  by  the  Naval  Explosive  Ordnance 
Disposal  Technology  Division  (NAVEODTECHDIV)  located  in  Indian  Head,  Maryland.  ODESA 
was  developed  to  address  the  Government's  need  for  reliable  and  cost-effective  UXO  Clearance 
Technology.  This  need  has  become  a  priority  within  the  Department  of  Defense  (DOD)  because  of 
military  downsizing,  reductions,  and  consolidations;  and  the  need  to  close,  clean-up  and  return  to 
the  public  military  bases  and  ranges.  Several  projects  performed  under  UXO-CTP  center  around  the 
demonstration  of  ordnance  detection  sensor  technology.  The  output  data  provided  by  sensors,  in 
combination  with  site  specific  and  environmental  information,  will  be  fused  together  by  ODESA  to 
provide  a  better  understanding  of  the  location  and  identification  of  buried  ordnance. 

The  ODESA  Project  was  started  as  a  multi-phase  effort  to  provide  a  fusion  mechanism  for 
sensor,  location,  environmental  and  site  specific  information.  This  data  fusion  process  is  intended 
to  enhance  the  effectiveness  of  a  single  sensor  system  in  the  location,  identification  and  classification 
of  buried  unexploded  ordnance.  As  a  result,  the  most  efficient  and  cost  effective  remediation 
mechanism  can  be  determined  for  each  survey  site.  The  goals  of  the  ODESA  Project  are  to  develop 
a  knowledge  based  expert  system  that  utilizes  state-of-the-art  data  fusion  techniques  such  as  neural 
networks  and  fuzzy  logic  to  address  the  buried  UXO  problem.  ODESA  processing  replaces  much 
of  the  manual  analysis  performed  by  sensor  experts  on  large  amounts  of  information  from  different 
sensors,  environments  and  sites.  Used  in  combination  with  other  processing  mechanisms,  ODESA 
can  learn  how  to  adapt  to  new  sensors,  sites  and  survey  environments.  ODESA's  processing  is 
modelled  after  a  human's  ability  to  reason,  interpret  and  correlate  sensor  information  obtained  from 
a  variety  of  sources. 

The  purpose  of  this  report  The  Evaluation  of  ODESA.  is  to  document  the  work  that  has  been 
performed  to  date  on  the  ODESA  project,  outline  the  strategy  developed  to  address  data  fusion 
aspects  of  the  buried  UXO  problem,  assess  the  feasibility  of  the  technology  to  improve  the  overall 
characterization  of  buried  UXO  and  provide  recommendations  for  future  efforts  in  the  data  fusion 
arena.  The  report  is  divided  into  several  sections  which  explain  the  UXO  problem  and  the  need  that 
started  UXO-CTP  and  ODESA  efforts.  The  report  documents  the  work  performed  under  each 
development  phase;  outlines  the  documentation,  design  and  fabrication  efforts;  provides  information 
on  the  ODESA  1.0  prototype;  outlines  the  criteria  used  to  evaluate  the  system  and  provides 
recommendations  for  future  work. 
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1.0  INTRODUCTION 


The  purpose  of  this  report  is  to:  (1)  document  the  work  that  has  been  performed  under  the 
Ordnance  Detection  Expert  Support  Application  (ODESA)  and  the  logic  behind 
Government  decisions  made  to  date;  (2)  evaluate  the  progress  and  feasibility  of  the  ODESA 
program  in  the  world  of  data  fusion,  knowledge  based  and  expert  systems;  and  (3)  provide 
this  report  as  a  tool  to  Naval  Explosive  Ordnance  Disposal  Technology  Division 
(NAVEODTECHDIV)  and  the  U.S.  Army  Environmental  Center  (USAEC)  decision  makers 
for  determining  whether  the  use  of  artificial  intelligence  is  feasible  for  the  detection, 
identification  and  remediation  of  buried  unexploded  ordnance  (UXO).  ODESA  is  a 
component  of  the  Unexploded  Ordnance  Clearance  Technology  Program  (UXO-CTP) 
sponsored  by  USAEC  and  managed  by  NAVEODTECHDIV.  This  report.  The  Evaluation 
of  ODESA.  provides  background  information  on  work  performed  under  Phase  I  research 
efforts  and  Phase  II  initial  fabrication  and  prototyping  efforts.  Phase  I  consisted  of 
researching  commercial  off-the-shelf  (COTS)  software  programs  and  generating  the 
following  documentation:  System  Design  Trade  Study,  Software  Design  Document,  and 
Software  Flow  Chart.  Phase  II  consisted  of  fabrication  of  the  ODESA  1.0  prototype, 
delivery  and  set-up  of  hardware,  installation  of  software,  and  the  generation  of  the  following 
documents:  Final  Technical  Report,  Software  User's  Manual,  and  COTS  Operations  Manual. 

This  report  is  organized  to  provide  the  reader  with  an  understanding  of  the  UXO  problem, 
the  objectives  of  UXO-CTP,  and  the  use  of  ODESA  to  meet  some  of  the  program  objectives. 
The  report  provides  the  reader  with  the  information  used  to  proceed  from  concept  to 
prototype,  documents  the  logic  behind  the  prototype  design,  provides  information  concerning 
the  basic  functions  of  the  ODESA  1.0  prototype,  and  evaluates  the  progress  of  the  ODESA 
effort  based  on  UXO-ATD  program  goals.  The  successfulness  of  the  ODESA  effort,  based 
on  established  goals,  is  discussed  and  recommendations  are  provided  concerning  where 
future  program  emphasis  should  be  placed  in  developing  program  strategies  related  to  UXO 
data  fusion  and  analysis. 


2.0  BACKGROUND 

With  the  new  emphasis  on  DOD  reductions  and  consolidation,  the  Federal  Government  has 
targeted  a  number  of  bases  and  military  ranges  for  closure,  clean-up  and  eventual  reclamation 
by  the  public.  These  military  installations  include  base  realignment  and  closure  (BRAG) 
sites,  formerly  used  defense  (FUD)  sites  and  installation  restoration  program  (IRP)  sites. 
Environmental  concerns  regarding  hazardous  materials  and  UXO  items  at  many  of  these  sites 
has  led  Congress  to  mandate  actions  toward  base  clean-up  operations.  As  an  initial  step 
toward  addressing  the  ordnance  and  hazardous  materials  clearance  problem, 
NAVEODTECHDIV  and  USAEC  hosted  the  1994  Jefferson  Proving  Ground  (JPG)  Phase 
I  Demonstrations  in  Madison,  Indiana.  JPG  Phase  I  was  open  to  government,  university  and 
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private  organizations  for  the  purpose  of  demonstrating  technologies  designed  to  detect, 
identify  and  remediate  buried  UXO.  JPG  I  consisted  of  two  controlled  test  sites:  a  40  acre 
test  site  for  ground  vehicular  and  man-portable  detection  and  remediation  systems  and  an  80 
acre  site  for  airborne  detection  systems.  Buried  inert  ordnance,  typical  range  debris,  false 
targets  and  simulated  hazardous  waste  containers  were  buried  and  surveyed  in  for  accurate 
location  of  test  items.  Public  solicitations  for  demonstrations  were  advertised  in  a  variety 
of  media  encouraging  sensor  developers,  academia,  range  clearance  companies  and 
government  agencies  to  participate  in  this  large  scale  demonstration.  Demonstrators  were 
tasked  with  surveying  either  the  40  or  80  acre  test  site  (depending  on  system  configuration), 
analyzing  the  sensor  information  obtained  from  these  tests  and  generating  a  target  data  set 
of  the  items  found  and  their  locations.  Additional  controlled-site  demonstrations  will  be 
performed  during  JPG  Phases  II  and  III  and  other  site  specific  demonstrations. 

The  results  of  the  JPG  Phase  I  demonstrations  were  used  as  a  baseline  in  determining  the 
performance  characteristics  of  "state-of-the-art"  detection  systems  and  where  the  government 
should  focus  its  research  and  development  (R&D)  efforts  in  the  future.  JPG  Phase  II  will  be 
open  to  demonstrators  whose  systems  were  not  mature  enough  to  perform  during  JPG  Phase 
I  and  to  demonstrators  that  have  made  significant  improvements  to  their  systems  since 
demonstrating  during  JPG  Phase  I.  The  results  of  the  JPG  Phases  II  and  III  demonstrations 
will  be  used  to  measure  the  progress  made  in  UXO  technology  over  the  past  year. 

In  addition  to  determining  the  UXO  detection,  identification  and  remediation  capabilities  of 
existing  systems,  NAVEODTECHDIV  pursued  sensor  and  data  processing  technology 
enhancements  under  the  UXO-CTP,  an  advanced  technology  demonstration  effort.  One 
purpose  of  the  UXO-CTP  is  to  develop  test-bed  systems  to  demonstrate  new  technologies 
in  the  area  of  UXO  detection,  identification  and  remediation  technologies  whose 
performance  characteristics  surpass  those  of  the  systems  demonstrated  at  the  JPG  controlled 
site  tests.  Current  efforts  focus  on  demonstrating  airborne  (Airborne  Ground  Penetrating 
Radar  -  AGPR) ,  ground  vehicular  (Subsurface  Ordnance  Characterization  System-SOCS) 
and  man-portable  systems  (Man  Portable  Ordnance  Detection  System  -  ManPODS)  using 
a  variety  of  sensors  and  sensor  types.  The  sensor  data  from  each  of  these  platforms  will  be 
analyzed  using  sensor  specific  processing  algorithms  and  formatted  in  a  standard  data  set 
(STD).  All  sensor  information  acquired  from  NAVEODTECHDIV's  UXO-CTP  efforts  will 
be  stored  a  standard  format  regardless  of  the  sensors  or  platforms  used;  this  will  allow  the 
government  to  scrutinize  data,  sensor  processing  techniques  and  abilities  professed  by  data 
analysis  companies.  Figure  1  is  a  block  diagram  that  provides  information  flow  between 
ODESA  and  other  external  UXO-CTP  external  subsystem  efforts.  This  diagram  outlines 
interactions  between  the  simultaneous  data  collection  and  processing  system  (SIDCAPS), 
explosive  ordnance  disposal  (EOD)  personnel,  user  input  files  and  other  UXO-CTP  efforts. 
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Figure  1 .  Information  Flow  Between  ODESA  and  External  Subsystems 


Sensors  currently  being  investigated  for  the  ground  vehicular  platform  (SOCS)  include: 
ground  penetrating  radar  (GPR),  cesium  vapor  magnetometers,  flux  gate  gradiometers,  tensor 
magnetometers,  induction  coils  and  inframetric  cameras.  The  testing  performed  by  SOCS 
will  not  be  limited  to  the  above  sensors,  but  will  evolve  with  the  development  of  new  UXO 
technologies.  The  sensor  being  investigated  for  the  airborne  system  is  a  ground  penetrating 
radar  and  the  sensors  being  investigated  for  the  manportable  system  are  ground  penetrating 
radar  and  gradiometers.  The  knowledge  acquired  from  processing  sensor  information  will 
be  used  to  enhance  existing  technologies  and  demonstrate  new  technologies,  and  this 
knowledge  will  be  transferred  to  the  private  sector  through  NAVEODTECHDIV's 
Technology  Transfer  Program.  It  is  the  Government's  intention  to  further  developments  in 
UXO  detection,  identification  and  remediation  technologies  by  aiding  and  encouraging 
private  industry  involvment  in  solving  the  buried  UXO  problem. 

Existing  single-sensor  detection  systems,  many  of  which  are  commercially  available,  are 
unable  to  accurately  and  reliably  detect  buried  UXO.  In  addition  to  the  actual  performance 
characteristics  of  detection  systems  such  as  sensor  accuracy  and  processing  routines,  the 
sensor’s  ability  to  detect  and  identify  buried  UXO  depends  upon  site  specific  conditions  such 
as  soil  type,  moisture  content,  ferrous  content,  terrain  type,  etc.  No  single  sensor  system  has 
been  found  that  could  adequately  address  the  buried  UXO  problem  by  providing  reliable  and 
accurate  buried  UXO  information.  As  a  result,  NAVEODTECHDIV  has  focussd  its  efforts 
on  demonstrating  multi-sensor,  multi-platform  systems  for  the  detection  of  buried  UXO  and 
multi-source  data  fusion  for  the  identification  and  characterization  of  buried  UXO. 
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Some  commercial  companies  utilize  the  multi-sensor  approach,  but  make  separate  surveys 
using  each  sensor  type.  NAVEODTECHDIV's  approach  is  to  simultaneously  obtain  data 
from  multiple  sensors  during  a  single  survey  operation  in  order  to  eliminate  the  time  and 
money  expended  from  performing  multiple  surveys.  The  information  acquired  from 
individual  sensor  surveys  is  typically  analyzed  by  an  individual  making  comparisons 
between  sensor  results;  the  performance  of  an  individual  sensor  or  system  thus  depends  on 
the  analyst's  knowledge  of  UXO,  the  performance  of  the  sensor  in  different  environments 
and  the  analysts  knowledge  (or  lack  of  knowledge)  of  site  history  and  ordnance  use.  To  fill 
the  multi-sensor  data  fusion  need  in  an  automated  approach,  eliminate  the  need  for  an 
individual's  analysis  of  sensor  information,  and  incorporate  topographic,  historical  and  site 
specific  information,  NAVEODTECHDIV  began  the  development  of  ODESA.  ODESA's 
function  is  to  perform  data  fusion  tasks  through  the  use  of  expert  systems,  neural  networks 
and  fuzzy  logic  approaches.  The  more  accurate  and  reliable  information  on  buried  UXO 
identification  and  location  (provided  by  ODESA)  will  be  provided  to  range  clearance  and 
base  clean-up  decision  makers  and  to  other  parties  responsible  for  site  remediation 
processes. 

In  a  concurrent  effort  to  address  the  UXO  remediation  need,  NAVEODTECHDIV  is 
developing  a  low  cost  autonomous  ordnance  excavator  (AOE)  for  autonomous  remediation 
of  buried  UXO,  through  Wright  Laboratory  at  Tyndall  Air  Force  Base.  The  AOE  effort 
builds  upon  previous  Air  Force  work  performed  under  the  Rapid  Runway  Repair  (RRR) 
Program.  When  the  cold  war  ended  and  there  was  no  need  for  rapid  runway  repair 
mechanisms,  RRR  technology  was  transitioned  to  the  UXO-CTP.  The  information 
generated  by  ODESA,  target  information  in  a  standard  data  set,  will  be  provided  to  the  AOE 
for  autonomous  excavation  of  selected  UXO  items  at  survey  sites.  The  AOE  autonomously 
plans  a  path  from  some  starting  point  to  the  item  of  interest,  navigates  to  that  point  and 
uncovers  the  item.  This  same  target  information  can  also  be  supplied  to  the  Explosive 
Ordnance  Disposal  (EOD)  field  troop  so  that  manual  remediation  efforts  can  be  performed 
in  situations  where  autonomous  or  tele-robotic  excavation  is  not  feasible. 


3.0  PURPOSE  OF  THE  ODESA  PROJECT 

The  development  of  the  ODESA  prototype  centered  around  the  Government's  need  to 
analyze  sensor  information  acquired  from  different  sensor  platforms  and  sensor  types  along 
with  historical,  topographical  and  user  input  information  to  achieve  better  detection  rates  and 
reduce  false  alarms  that  directly  affect  the  scope,  cost  and  ultimately  the  Government's 
decision  to  remediate  a  site.  ODESA  is  a  tool  to  achieve  this  goal.  Sensor  data  can  be 
acquired  at  different  times,  with  different  systems  and  with  different  environmental  factors 
influencing  sensor  effectiveness  and  performance.  ODESA's  processing  emphasis  was  to 
combine  neural  networks,  fuzzy  logic  processing,  genetic  algorithms  and  heuristic 
probabilistic  analyses  in  an  expert  system  shell  to  provide  the  most  accurate,  effective  and 
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reliable  information  on  the  location,  identification  and  classification  of  buried  UXO. 
ODESA's  function/purpose  was  to  provide  the  data  fusion  function  that  would  combine  all 
of  the  information  obtained  from  UXO-CTP  sensor  test-bed  systems  into  a  single  more 
accurate  target  report.  The  development  of  the  ODES  A  prototype  version  1.0  was  performed 
by  PRC  Inc.  located  in  McLean,  Virginia  under  two  separate  phases:  Phase  I  consisted  of 
researching  COTS  software  that  could  be  integrated  together  to  form  ODES  A,  developing 
hardware  and  software  designs  and  design  documentation  and  generating  all  required 
planning  documents;  Phase  n  consisted  of  fabricating  ODESA  1.0  and  preparing  associated 
documentation. 


3.1  Phase  I  -  System  Design  Trade  Studies  and  Initial  Investigations 

Phase  I  consisted  of  investigating  COTS  products  to  address  the  data  fusion  and  analysis, 
defining  the  functional  requirements  of  the  ODESA  system,  determining  the  design  drivers 
of  a  prototype  system,  determining  the  software  technical  design  requirements  and 
developing  recommendations  for  the  optimal  configuration  of  the  ODESA  prototype. 
Investigations  included  researching  data  communication  tools,  artificial  neural  network 
products,  fuzzy  expert  systems,  graphical  user  interfaces  (GUI),  object  database  management 
systems,  computer  hardware  and  developing  recommendations  for  the  hardware  and  software 
architecture  of  the  initial  prototype.  A  System  Design  Trade  Study,  Software  Development 
Plan  and  Software  Flow  Chart  were  generated  for  the  development  of  ODESA;  these 
documents  outlined  the  strategy  for  prototype  development. 

Note:  The  following  information  was  generated  under  Phase  I  efforts  and  was  obtained 
from  the  System  Design  Trade  Study,  Software  Development  Plan  and  Software  Flowchart 
Documents.  Specifics  on  these  documents  are  included  in  the  reference  section. 

As  a  result  of  ODESA  Phase  I,  the  following  functional  requirements  were  defined: 

3.1.1  Functional  Requirements 

1.  ODESA  shall  be  able  to  retrieve  and  use  sensor  and  navigation  information 
from  multiple  systems,  regardless  of  method  of  employment,  stored  in 
standardized  data  sets  (STDs). 

2.  ODESA  shall  use  site  and  environmental  information  including  data  stored 
in  historical,  geographical  and  geological  databases;  data  obtained  on  known 
anomalies  and  additional  user  input  to  increase  the  probability  of  detection 
and  reduce  the  false  alarm  rate. 

3.  ODESA  shall  make  intelligent  decisions  based  on  ordnance  characteristics 
such  as  location,  buried  depth,  classification,  orientation  and  size. 
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4.  ODESA  shall  be  capable  of  automatically  downloading  sensor  and  site 
information  or  of  manually  input  by  an  operator. 

5.  ODESA  shall  provide  output  data  in  STD  format  usable  by  automated, 
remote  or  manual  remediation  methods. 

6.  ODESA  shall  provide  output  in  map  or  tabular  format  usable  by  EOD 
technicians. 

7.  ODESA  users  shall  be  able  to  modify  historical  target  files  after  ordnance 
items  have  been  exposed  in  order  to  maintain  up-to-date  historical  files  of 
surveyed  areas  and  provide  a  feedback  loop  to  improve  system  performance 
and  predictions. 


System  design  drivers  were  also  generated  for  ODESA  1 .0  during  Phase  I  efforts.  These 
design  drivers  were  used  to  address  the  functional  requirements  specified  above  and  as  a 
basis  for  developmental  work.  These  drivers  are  summarized  below. 


3.1.2  Design  Drivers 

1 .  Create  Test  Bed:  ODESA  is  a  test-bed  computer  system.  Its  purpose  is  to 
explore  a  variety  of  processing  methods  to  obtain  the  optimal  configuration 
for  solving  the  unexploded  ordnance  detection  problem. 

2.  Apply  Advanced  Techniques:  ODESA  will  make  intelligent  decisions  with 
a  combination  of  neural  network,  fuzzy  logic,  heuristic  probabilistic,  and 
genetic  algorithm  processing  techniques. 

3.  Add  Value:  ODESA's  performance  is  based  on  the  value  that  ODESA 
processing  adds  to  the  UXO  location,  identification  and  classification 
predictions  of  the  individual  sensor  devices/systems. 

4.  Utilize  Existing  Software:  ODESA  hardware  and  software  architecture  will 
minimize  software  development  times  and  costs  by  utilizing  COTS  products. 

5.  Comply  With  Industry  Standards:  ODESA  will  comply  with  industry 
standards  for  operating  systems,  development  languages,  network  computing, 
peripherals,  software  and  other  components. 
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6.  Extendability:  A  flexible  ODES  A  system  design  requires  a  software  and 
hardware  architecture  that  will  support  system  development,  product 
integration,  demonstration,  and  fielding.  The  architecture  should  support 
future  and  emerging  technologies  so  that  the  system  can  learn  and  evolve 
over  time. 

7.  Usability:  ODESA  should  take  advantage  of  the  superior  pattern-matching 
and  classifying  capabilities  of  the  sensor  experts.  All  user  interface  tools 
should  be  easy  to  use  and  involve  the  analyst  in  meaningful  ways. 


Software  Technical  Design  Requirements  for  the  ODESA  software  architecture  were  derived 
from  the  functional  requirements  and  design  drivers  generated  under  Phase  I  efforts.  The  key 
software  architecture  technical  requirements  were  determined  to  be: 


3.1.3  Software  Technical  Design  Requirements 

1 .  Performance:  No  real-time  requirements  exist  for  ODESA  data  processing; 
however,  because  of  the  highly  interactive  nature  of  the  prototype 
architecture,  ODESA  must  have  a  good  level  of  performance. 

2.  Extendability:  ODESA  must  be  able  to  utilize  and  process  data  acquired 
from  new  sensor  types  with  minimal  changes  to  the  existing  system. 

3.  Open  Architecture:  The  system  architecture  must  minimize  the  difficulty  of 
interfacing  new  components.  ODESA  must  be  capable  of  incorporating  and 
trying  out  new  software  with  minimal  changes  to  the  existing  system. 

4.  Usability:  The  ODESA  operator  must  be  able  to  easily  interact  with  the  the 
ODESA  system.  Where  possible,  the  user  interface  for  ODESA  should  not 
complicate  user  interaction. 

5.  Reliability:  ODESA  must  be  constructed  of  mature,  technically  supportable 
COTS  tools  and  high  quality,  fully  documented  custom  software. 

6.  Compliant:  To  ensure  ease  of  adding,  replacing  or  modifying  hardware  or 
software  components,  ODESA  will  be  compliant  with  existing  and  emerging 
standards. 
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3.1.4  Recommendations  for  ODESA  Prototype  Version  1.0  Architecture 


As  a  result  of  Phase  I  efforts,  PRC  provided  information  on  a  variety  of  hardware  and 
software  design  options  for  the  ODESA  prototype;  a  final  hardware  and  software 
architecture  recommendation  for  the  prototype  ODESA  system  was  provided  in  the 
ODESA  System  Design  Trade  Study.  Approval  of  the  recommended  approach  by 
NAVEODTECHDIV  gave  PRC  authorization  to  begin  Phase  n  efforts.  The 
following  contains  recommendations  presented  by  PRC  for  the  ODESA  1.0 
prototype: 

Input  Data  Model 

Initial  input  data  to  the  ODESA  prototype  should  consist  of  target  data  produced  by 
individual  sensor  systems  and  stored  in  an  STD.  This  will  allow  the  ODESA 
development  to  concentrate  on  data  fusion,  and  let  the  need  for  additional  data  be 
driven  by  lessons  learned  obtained  from  different  processing  techniques. 


Hardware  Architecture 

The  proposed  hardware  architecture  consists  of  a  homogenous  network  that  will 
initially  contain  a  single  mid-level  workstation  that  will  be  upgraded  to  two  mid¬ 
level  workstations  for  semi-raw  data  analysis.  The  initial  recommendation  is  to  use 
a  SUN  SPARC  20  for  the  ODESA  1.0  prototype  which  analyzes  target  data  sets  only. 
A  two  computer  network  is  recommended  for  higher  versions  of  ODESA  when  semi¬ 
raw  data  analysis  algorithms  are  developed  and  utilized. 

The  hardware  recommendation  is  based  on  the  selection  of  an  architecture  that 
supports  multiple,  cooperating  hardware  platforms.  Such  platforms  will  allow  the 
development  of  a  system  that: 

#  Meets  the  ODESA  performance  requirements  for  all  stages  of  the  life-cycle. 

#  Is  flexible  enough  to  support  integration  of  COTS  components  that  may  be 
limited  to  certain  platforms.  Future  specialized  components  could  thus  be 
integrated  by  adding  additional  hardware  as  necessary. 

#  Serve  as  a  test-bed  for  emerging  processing  techniques. 


Software  Architecture 

The  major  considerations  for  the  software  architecture  option  are  data 
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communications,  artificial  neural  network  capabilities,  fuzzy  expert  system,  GUI  and 
the  object  database  management  system.  The  recommended  system  utilizes  a  series 
of  independent  applications  sharing  common  information  across  a  distributed 
computing  platform.  Figure  3-1  illustrates  this  software  architecture  approach.  The 
client/server  based  approach  provides  a  modular,  expandable  solution  that  forsees  the 
incremental  inclusion  of  specialized  applications  to  the  system.  To  facilitate  this 
approach,  a  COTS  distributed  object  database  management  system  will  also  be 
selected  to  provide  the  common  data  storage  and  data  communication  tools  needed 
in  a  client/server  environment. 


Figure  2.  Software  Architecture 


User  Interaction  Model: 

A  mid-level  user  interaction  model  is  recommended  that  provides  a  simple  load-and- 
go  mechanism  with  additional  data  visualization  tools.  The  key  user  interface 
interactions  recommended  involve  the  ability  to  view  and  manipulate  target  data,  site 
data  and  other  data  in  two  and  three-dimensiona.  To  ensure  that  user  interface 
features  can  be  added,  the  Motif  library  approach  is  recommended  over  the  COTS 
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Data  Visualization  Tool.  Figure  3  depicts  a  block  diagram  of  the  user  interaction 
model  recommended  under  this  effort. 


Figure  3.  User  Interaction  Model  for  ODESA. 

Recommended  Data  Analysis  Model: 

Figure  4  depicts  several  of  the  sensor  data  analysis  approaches  discussed  in  the 
ODESA  System  Design  Trade  Study.  The  application  of  intelligent  computing  to 
fuse  and  correlate  target  data  from  individual  sensors  can  be  implemented  in 
unlimited  combinations.  Figure  4  provides  several  different  data  analysis  schemes 
including:  Model  1  -  Fuzzy  Expert  Systems;  Model  2  -  Single  Neural  Network 
System;  Model  3  -  Conventional  Expert  Systems;  Model  4  -  Single  neural  network 
for  each  sensor  type  that  improves  the  target  list  and  is  then  applied  to  the  fuzzy 
expert  system;  Model  5  -  A  series  of  neural  networks  per  sensor,  each  specializing 
in  a  particular  type  of  target  identification  whose  outputs  are  then  fused  by  a  master 
neural  network  and  then  passed  onto  a  fuzzy  expert  system.  Model  4  was  the  data 
analysis  configuration  recommended  by  PRC. 

The  recommended  data  analysis  model  is  a  combination  of  neural  networks,  fuzzy 
logic  and  expert  systems.  This  approach  utilizes  neural  networks  to  improve  the 
accuracy  of  the  target  data  produced  by  individual  sensors.  This  will  allow  ODESA 
to  learn  from  collected  data  signatures.  For  each  sensor  type,  a  neural  network  will 
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be  developed  to  improve  sensor  output  in  the  form  of  an  STD.  A  fuzzy  expert  system 
will  then  fuse  the  predicted  target  data  from  each  neural  network  to  identify  and 
locate  UXO.  The  combination  of  a  conventional,  rule-based  expert  system  and  a 
fuzzy  logic-based  expert  system  is  recommended  because  of  the  mixed  quality  of  the 
information  that  will  be  embedded  in  the  system.  Both  heuristics  from  human 
experts  and  patterns  learned  from  data  will  be  available  for  ODES  A  utilization,  the 
former  represented  by  the  expert  system,  the  latter  by  the  fuzzy  component. 


Figure  4. 

Data  Analysis  Models 


This  page  is  intentionally  left  blank 
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3.2  Phase  II  -  Development  of  ODESA  1.0  -  Standard  Data  Set  Analysis 


Based  on  Phase  I  research,  investigations  and  recommendations  NAVEODTECHDIV  and 
USAEC,  authorized  initial  development  of  ODESA  through  Phase  11  efforts.  The  scope  of 
Phase  n  consisted  of  developing  an  initial  prototype  using  STD  processing  techniques  only 
Historical,  site  specific  and  semi-raw  data  information  will  be  incorporated  into  the  data 
fusion  approach  during  later  program  phases.  The  purpose  of  Phase  n  was  to  build  a  limited 
prototype  system,  with  GUI,  utility  and  analysis  capabilities.  NAVEODTECHDIV  wanted 
to  demonstrate  that  target  information  acquired  from  several  different  systems  could  be  fused 
to  provide  more  accurate  and  reliable  information  than  could  be  obtained  from  the  analysis 
of  data  from  any  single  sensor  system.  NAVEODTECHDIV's  intention  was  to  add 
additional  analysis  capabilities  like  semi-raw  data  analysis,  incorporation  of  historical, 
topographical  and  site  specific  information  and  a  more  enhanced  graphical  user  interface,  to 
ODESA  in  separate  phases  based  on  the  success  of  standard  data  set  analysis. 

Phase  n  efforts  consisted  of  development  of  data  fusion  algorithms  and  a  hardware  and 
software  architecture  that  would  support  the  algorithms.  Several  different  analysis 
techniques  were  investigated  including,  genetic  algorithms,  heuristic  and  probabilistic 
methods,  neural  networks  and  fuzzy  logic  applications.  For  Phase  H,  the  processing  of 
standard  data  set  information  only,  the  two  data  processing  approaches  chosen  were  genetic 
algorithms  and  heuristic/probabilistic  analysis.  The  neural  network  and  fuzzy  logic 
approaches  were  reserved  for  semi-raw  data  analysis  under  Phase  HI,  where  the  full 
development  of  an  expert  system  can  take  advantage  of  these  higher  level  processing  tools. 

The  development  of  the  hardware  and  software  to  support  the  data  analysis  tools  consisted 
of  the  generating  the  GUI,  limited  utility  capabilities,  the  generation  of  an  object  oriented 
software  application  and  the  definition  of  the  JPG  test  site  in  terms  of  location,  topography 
and  grid  orientation.  Figure  5  provides  a  block  diagram  drawing  of  the  user  interface 
information  flow  for  ODESA;  it  outlines  the  interconnectability  ODESA  software 
components  through  the  user  interface. 

The  sensor,  site  and  demonstrator  information  supplied  to  PRC  for  Phase  n  efforts,  was 
drawn  from  the  1994  JPG  Phase  I  demonstrations.  The  demonstrator  information  obtained 
from  the  JPG  Phase  I  demonstrations  was  used  as  input  to  ODESA.  In  addition, 
approximately  half  of  the  target  key  (information  on  the  actual  locations  of  the  buried  UXO 
items)  was  supplied  to  ODESA  as  a  training  set  to  teach  the  expert  system  how  to  correctly 
analyze  sensor  information.  PRC  developers  performed  a  test  to  determine  how  well 
ODESA  could  fuse  information  from  four  specific  demonstrators  (ADI,  Coleman,  Geo- 
Centers  and  UXB)  that  represented  ground  penetrating  radar  and  magnetometer  technologies. 
Like  the  demonstrators  at  JPG  I,  ODESA  generated  target  information  that  was  also  supplied 
to  the  Government's  JPG  Phase  I  evaluation  mechanism  for  scoring  Automated  Research 
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Systems  (ARS)  and  Institute  for  Defense  Analysis  (IDA)  algorithms,  based  on  these  four 
demonstrators.  This  impartial  mechanism  determined  that  ODESA  could  provide  a  more 
accurate  accounting  of  buried  UXO  by  reducing  the  amount  of  "false  alarms"  detected, 
where  false  alarms  is  defined  as  detected  anomalies  indicated  as  UXO  items  that  were  in 
actually  either  non-UXO  items  or  non-existent  targets.  Since  ODESA's  detection  ability  can 
only  be  as  good  as  the  sensor  information  it  is  provided,  detection  rates  of  ODESA  in 
comparison  to  the  best  JPG  I  detection  system  would  not  be  significant;  however,  it  was 
determined  that  ODESA  could  minimize  the  false  alarm  rates  experienced  by  individual 
sensors  by  utilizing  the  best  performance  characteristics  of  each  sensor. 
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Figure  5.  User  Interface  Information  Flow 
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The  first  tests  performed  by  ODESA  utilized  the  scoring  routine  to  characterize  system  and 
demonstrator  performance.  Figure  6  below  depicts  the  performance  of  fifteen  target  data  sets 
from  fifteen  separate  demonstrators  (identified  as  dl  -  dl5)  using  the  scoring  mechanism  and  a 
four  foot  search  radius.  The  three  metrics  used  for  comparison  were  percentage  of  targets 
detected,  percentage  of  false  reports  and  percentage  of  the  area  surveyed. 


Figure  6.  Demonstrator  Report  Percentage  with  4-Foot  Search  Radius 


The  percentage  of  targets  detected  criteria  was  calculated  by  taking  the  total  number  of  detections 
obtained  by  a  demonstrator  divided  by  the  number  of  possible  targets.  The  percent  of  false 
reports  criteria  was  calculated  as  the  number  of  false  reports  divided  by  the  total  number  of 
reports.  In  addition,  the  total  number  of  detections  and  false  reports  for  the  same  target  data  sets 
(dl-dl5)  was  calculated  and  is  presented  in  Figure  7.  This  figure  is  provided  to  give  the  reader  an 
understanding  of  the  enormous  number  of  reports  involved  in  the  fusion  process,  and  the  need  for 
efficient  data  processing  and  manipulation  software. 
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Figure  7.  Demonstrator  Report  Counts  with  a  4-Foot  Search  Radius 


Based  on  these  scoring  techniques,  ODESA  developers  could  judge  how  well  the  ODESA  data 
fusion  techniques  were  performing  in  comparison  to  the  data  analysis  techniques  of  other 
demonstrators.  This  scoring  routine  is  different  than  the  techniques  used  by  ARS  and  IDA  to 
judge  the  performance  of  JPG  I  demonstrators.  Figure  8  depicts  the  results  of  several  ODESA 


data  fusion  attempts. 


Figure  8.  ODESA  Percentage  Results  for  Several  Different  Algorithms 
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The  target  data  set  known  as  "ga"  was  produced  by  the  Genetic  Algorithm  Module  (GAM)  based 
on  rules  created  by  training  the  genetic  algorithm  using  four  government  selected  data  sets.  The 
remaining  data  sets  depicted  in  the  figure  were  produce  by  the  Heuristic  Probabilistic  Module 
(HPM)  by  varying  the  target  data  sets  to  fuse  and  the  likelihood  threshold.  For  example,  the  data 
set  "afgu  80"  represents  a  fusion  of  data  sets  "a,"  "f,"  "g,"  and  "u"  using  a  likelihood  threshold  of 
80  percent.  Demonstrator  data  sets  were  given  letter  descriptors  like  "a"  and  "f '  to  eliminate  any 
bias  that  could  come  out  of  knowing  what  company  that  demonstrator  data  set  represented.  An 
examination  of  the  above  figure  shows  that  the  percentage  of  detections  and  false  reports 
increases  with  a  decreasing  likelihood  threshold. 
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Figure  9.  ODESA  Counts  Results  for  Varying  Algorithms 

The  same  type  of  chart  as  shown  in  Figure  7  for  actual  number  of  detected  targets  and  reports  for 
the  same  set  of  ODESA  target  data  sets  is  depicted  in  Figure  9.  As  discussed  in  the  ODESA  Final 
Technical  Report  for  Phase  II,  the  report  percentages  varied  according  to  search  radius.  Figure 
10  shows  how  well  the  ODESA  processing  algorithms  described  above  compared  to  the 
demonstrator  target  reports.  The  results  of  the  ODESA  cbmparisons  are  less  reported  false 
alarms;  the  false  alarms  are  minimized  during  the  data  fusion  process.  The  best  performance 
characteristics  from  each  sensor  system  is  combined  in  the  ODESA  processing. 


Specifics  concerning  the  results  of  the  ODESA  processing  can  be  obtained  by  reviewing  the 
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ODESA  Final  Technical  Report  for  Phase  II.  Details  about  this  document  can  be  found  in  the 
Reference  section  of  this  report. 


Figure  10.  ODESA  and  Demonstrator  Report  Percentages  Compared 

The  results  obtained  from  the  ARS  scoring  can  not  be  interpreted  in  the  same  way  as  the  results 

of  the  ODESA  processing.  There  are  several  specific  differences  between  the  ARS  and  the 

ODESA  scoring  mechanisms.  These  differences  are  outlined  below; 

#  The  ARS  method  lowers  detection  counts  since  a  single  demonstrator  report  can  match 
one  and  only  one  baseline  target. 

#  The  ODESA  Score  Utility  matches  a  single  demonstrator  report  with  as  many  baseline 
targets  that  are  within  the  critical  distance  of  the  report. 

#  The  ARS  method  raised  False  Negative  Ratios,  since  only  one  demonstrator  report  can 
match  a  single  baseline  report,  other  reports  within  the  critical  distance  are  counted  as 
false  negatives. 
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•  The  ODES  A  False  Report  percentage  will  be  lower  than  the  ARS  False  Negative 
Ratio,  since  multiple  demostrator  reports  can  match  a  single  baseline  target  or  known 
anomaly  and  are  not  considered  false. 


3.3  Phase  III  -  Enhanced  ODESA  Development  -  Semi-Raw  Data  Analysis 

This  report  is  intended  to  be  a  stepping  stone  for  further  ODESA  development.  Based  on  the 
success  of  Phase  n  efforts  (standard  data  set  or  target  data  set  analysis)  and  the  prospects  of 
further  ODESA  development,  Phase  IH  efforts  would  center  on  enhancing  ODESA's 
capabilities  in  the  realm  of  semi-raw  data  analysis.  NAVEODTECHDIV  intended  to  utilize 
neural  network  and  fuzzy  logic  capabilities  for  the  processing  of  semi-raw  data, 
topographical,  historical,  site-specific  information  and  performance  characteristics  of  sensors 
in  varying  environments  early  on  in  the  effort.  Since  program  emphasis  changed  from  that 
of  analyzing  semi-raw  data  sets  and  environmental/site  information  to  that  of  analyzing 
standard  data  sets  in  Phase  H,  initial  analyses  were  performed  on  target  data  only.  It  was  not 
feasible  to  start  developing  neural  network  and  fuzzy  logic  processing  routines  until  semi¬ 
raw  data  could  be  utilized.  The  semi-raw  data  and  historical  information  enhancements 
proposed  for  Phase  HI  would  allow  analysis  of  sensor  data  from  varying  test  sites, 
demonstrators  and  sensor  types. 

Plans  for  further  ODESA  development  include  utilizing  an  artificial  neural  network  and 
expert  system  approach  to  analyzing  semi-raw  sensor  data.  In  Phase  I  it  was  determined  that 
analysis  of  information  from  variety  of  sensor  types,  sensor  systems  and  platforms  required 
an  automated  approach.  ODESA's  software  design  was  generated  to  simulate  the  processes 
a  human  brain  performs  in  analyzing  information.  One  of  the  goals  of  the  ODESA  project 
was  to  develop  a  system  that  could  be  used  by  other  UXO-CTP  projects  that  could  take 
advantage  of  ODESA  neural  network,  fuzzy  logic  and  expert  system  development  for  UXO 
detection.  ODESA  could  then  "learn"  the  effects  of  soil  type,  environmental  conditions,  and 
site  specific  information  on  sensor  performance  and  provide  users  with  a  more  accurate 
accounting  of  buried  UXO  present  at  a  particular  site.  In  the  next  version  of  ODESA,  the 
neural  network  component  would  learn  to  classify  input  patterns  by  being  trained  on 
examples  and  classification  of  input  data ,  much  like  a  human  operator  does  when  comparing 
target  reports  from  different  sensors.  Neural  networks  can  be  used  singly  or  in  combination 
to  perform  a  variety  of  processing  tasks.  Individual  neural  networks  might  be  used  to 
discriminate  between  ordnance  and  clutter,  classify  suspected  ordnance  into  sizes  or  types, 
or  estimate  the  depth  or  orientation  of  buried  objects  from  the  patterns  and  convergence  of 
signal  strengths. 
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Neural  networks  are  a  type  of  computer  architecture  modeled  after  the  structure  and 
operations  of  the  human  brain.  In  contrast  to  conventional  computers,  neural  nets  have  no 
separate  processors,  memory  or  stored  software  programs,  instead  they  consist  of  a  large 
number  of  simple  processors  joined  together  by  a  complex  set  of  interconnections.  These 
networks  are  not  programmed,  but  are  "taught"  by  presenting  examples  of  material  to  be 
learned.  Figure  1 1  presents  an  illustration  of  how  the  individual  nodes  are  connected. 
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Figure  1 1 .  Diagram  of  Neural  Network  Design. 

An  expert  system  component  would  be  the  backbone  of  the  ODESA  design  stmcture.  It  will 
provide  the  system  with  a  means  of  reasoning,  an  would  allow  ODESA  to  correlate,  fuse  and 
act  upon  all  types  of  information  made  available  to  the  system.  Although  neural  networks 
can  provide  valuable  assistance  in  interpreting  sensor  inputs,  they  cannot  determine  the 
significance  of  their  findings  in  terms  of  the  total  mission  of  the  system.  This 
"interpretation"  function  must  be  performed  by  the  rule-based  logic  of  an  expert  system. 
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Under  Phase  III,  the  expert  system  component  would  provide  the  following  functions: 

•  Grouping  raw  data  points  based  on  factors  such  as  proximity  and  similarity. 

•  Classifying  the  groups  into  categories  relevant  to  the  ODESA  mission.  This 
function  would  assign  attributes  based  on  spatial  and  statistical  properties. 

•  Examining  attributes  and  classifying  them  as  belonging  to  one  or  more 
categories. 

•  Fusing  the  results  of  all  the  processing  applications  based  on:  the  area  under 
consideration,  sensor  readings,  buried  utilities  in  the  region,  history  of  the 
site,  etc. 

NAVEODTECHDIV  intends,  with  approval  to  continue  ODESA  development,  to  enhance 
the  target  data  processing  capabilities  of  the  ODESA  1.0  prototype  through  the  development 
of  neural  network  and  expert  system  processes  for  the  analysis  of  semi-raw  sensor  data.  The 
information  on  Phase  HI  efforts  is  provided  to  allow  reader  understanding  of  project  strategy, 
the  components  of  the  project  selected  for  demonstration  under  ODESA  1.0  and  the 
components  designated  as  ODESA  2.0  concepts.  The  future  of  the  ODESA  effort  depends 
on  sponsor  approval  to  continue  development. 


4.0  ODESA,  THE  SYSTEM  (VERSION  1.0) 

In  this  section,  information  will  be  provided  that  documents  the  current  configuration  of  the 
ODESA  1.0  prototype.  The  key  components  of  ODESA  1.0  (the  user  interaction  model,  data 
display  and  manipulation  capabilities,  database  management  tools,  analysis  algorithms)  are 
explained  in  detail  so  that  the  project  status  can  be  evaluated  in  Section  5.0. 

4.1  Background: 

The  goal  for  ODESA  1.0  was  to  intelligently  fuse  unexploded  ordnance  target  reports  from 
multiple  sensors  and  systems  to  produce  a  remediation  target  list  that  better  identifies  the 
location,  orientation  and  identification  of  buried  ordnance  and  clutter.  This  data  fusion  is 
accomplished  through  the  use  of  target  reports  only,  not  through  direct  analysis  of  raw  or 
semi-raw  sensor  information.  ODESA  1 .0  was  developed  in  parallel  with  the  data  collection 
efforts  at  the  JPG  Phase  I  demonstratiolns.  Many  of  the  JPG  Phase  I  demonstrator  data  sets 
utilized  by  the  ODESA  algorithms  had  to  be  "fine-tuned"  manually  for  insertion  into  the 
prototype  system.  As  a  result,  ODESA  1.0  is  a  proof  of  concept  analysis  tool  for  a  single 
site,  Jefferson  Proving  Ground.  The  main  objective  for  ODESA  1.0  was  to  maximize  the 
probability  of  detection  and  minimize  false  alarm  rates  of  demonstrator  supplied  sensor 
information  through  fusion  techniques.  When  the  NAVEODTECHDIV  STD  and  SRD 
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formats  are  finalized,  ODESA's  input  processes  can  be  modified  to  utilize  sensor  and  site 
specific  information  obtained  from  any  site  and  any  sensor  type.  ODES  A  1.0  is  not  a  turn¬ 
key  system  and  developers  assumed  that  ODESA  operators  will  be  familiar  with  the  basics 
of  SUNSPARC  20  operations  and  the  UNDC  environment. 

The  ODESA  1.0  prototype  was  developed  by  combining  features  from  several  commercially 
available  software  packages  with  custom  integration  software  developed  by  the  ODESA 
team.  These  COTS  packages  were  used  to  develop  the  GUI,  the  two-dimensional  and  three- 
dimensional  graphics  tools  in  the  user  interface  and  the  object  database  management  system. 
Initial  ODESA  development  was  been  accomplished  on  a  SUN  SPARC  20,  a  dual  processor 
UNIX  workstation.  The  SUN  SPARC  runs  under  the  Solaris  2.3  operating  system.  The 
optimum  COTS  products  to  be  utilized  by  ODESA  were  researched  in  Phase  I,  and  were 
proposed  in  the  System  Design  Trade  Study. 


4.2  User  Interaction  Model 

The  ODESA  1.0  user  interaction  model  (UIM)  defines  how  the  system  should  characterize 
the  user's  interactions;  focus  on  what  the  user  sees  and  what  he  needs  to  do  rather  than  how 
the  system  accomplished  the  specified  tasks.  The  UIM  provides  direction  for  GUI  design,  the 
tasks  the  user  must  perform,  and  the  flow  of  information  between  ODESA  and  the  user 
interface.  Although  all  aspects  of  the  UIM  have  not  been  developed  to  date,  the  framework 
of  this  model  is  already  in  place. 

4.2.1  Graphical  User  Interface 

ODESA  utilizes  an  X  Window  System  in  combination  with  the  OSF/Motif  Window 
Manager  to  provide  GUI  and  display  ODESA  information  to  the  user. 

X  Windows  is  a  set  of  subroutines  that  provide  an  interface  between  graphical 
devices  and  user  applications.  OSF/Motif  is  a  standard  workstation  GUI  used  in 
private  industry.  OSF/Motif  provides  a  Motif  Window  Manager  mechanism  that 
allows  the  user  to  manipulate  and  simultaneously  display  multiple  windows  on  the 
screen.  User  functions  include  the  ability  to  change  keyboard  settings,  focus 
themouse  pointer  and  control  windows  operations. 

The  ODESA  1.0  GUI  allows  the  users  to  access  all  functions  necessary  to  ODESA 
operations  from  the  main  ODESA  Window.  From  the  main  window,  the  user  can 
load  site  data;  display  target  sets;  review  information  about  individual  targets,  target 
sets,  sensor  and  organizations;  view  graphical  site  and  target  data  in  two  and  three 
dimensions;  and  perform  analyses  on  the  target  data.  Most  of  the  GUI  has  already 
been  developed;  items  that  have  not  been  developed  have  been  placed  as  "markers" 
in  the  windows  environment  so  additional  modules  can  be  incorporated  in  the  future. 
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4.2.2  User  Defined  Tasks 


The  UIM  outlines  the  tasks  a  user  must  perform  and  the  sources  of  knowledge  that 
must  be  utilized  to  perform  the  tasks.  The  UIM  also  includes  information  on  the 
relationships  between  tasks  and  the  expected  frequency  of  the  given  task.  Since 
ODES  A  1.0  was  a  proof-of-concept  prototype  to  demonstrate  that  better  target 
information  can  be  acquired  through  data  fusion,  the  ODESA  framework  was  built 
around  the  JPG  Phase  I  site  information  and  its  demonstrators.  NAVEODTECHDIV 
plans  to  make  the  analysis  routines  more  open  in  the  next  phase  of  development  by 
allowing  new  sites,  new  demonstrators  and  new  sensor  information  to  be 
incorporated  as  inputs  to  the  system.  The  following  is  a  list  of  user  tasks  that  will  be 
incorporated  into  future  versions  of  ODESA: 

#  Site  Definition  Task  -  produces  the  basic  definition  of  a  site  including 
geography,  topography,  roads,  utilities,  soil  types,  ordnance  types,  etc. 

#  Sensor  Survey  Review  Task  -  defines  the  information  concerning  sensors 
employed,  area  surveyed,  hot  spots,  missed  areas,  etc. 

#  Data  Analysis  Task  -  occupies  the  bulk  of  the  user's  time.  The  main  goal  is 
to  examine  sensor  information  and  refine  target  lists.  The  user  will  use  data 
visualization  software  in  conjunction  with  the  expert  system  and  neural 
network  components  to  perform  this  task. 

#  Target  Identification  List  -  provides  a  detailed  listing  of  pertinent  infonhation 
about  the  target,  evidence  supporting  its  identification  and  the  user's 
categorization  of  the  target. 

#  Remediation  Target  List  -  the  primary  output  of  ODESA  is  a  predicted  target 
list  (STD)  that  can  be  used  by  remediation  resources  to  excavate  the  UXO 
identified  by  ODESA. 

#  Remediation  Feedback/Results  Task  -  Once  a  target  is  excavated,  the  results 
of  that  remediation  effort  are  provided  to  ODESA  via  a  (STD)  input  from  the 
remediation  source.  The  information  can  then  be  fed  back  into  ODESA  so 
that  the  system  can  improve  its  performance  as  experience  is  gained  in  the 
remediation  cycle. 
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4.3  Data  Display  and  Manipulation: 

ODES  A  utilizes  a  graphical  display  program  called  PV-WAVE  that  provides  a  versatile  set  of 
commands  for  manipulating  and  displaying  data  in  a  graphical  format.  This  information  is  stored 
in  libraries  that  can  be  utilized  to  manipulate  display  information  in  several  ways  including  two- 
dimensional  and  three-dimensional  displays,  data  filtering,  rotation  of  data  and  slicing  of  data. 
PV-WAVE  can  accomodate  the  manipulation  of  large  amounts  of  data. 

Some  of  the  capabilities  of  the  PV-WAVE  software  package  are  illustrated  in  Figure  12,  a  screen 
capture  image  of  ODESA.  This  figure  depicts  an  image  of  a  test  area  in  the  three-  dimensional 
mode.  The  user  is  free  to  rotate  the  image,  zoom  in  on  a  particular  area  of  interest,  overlay 
topographical  and  contour  lines  and  change  the  colors  of  the  three  dimensional  image  to  allow  for 
easier  viewing  of  suspected  targets. 
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Figure  12.  ODESA  Site  Display  Window 
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test  area  and  Figure  14  depicts  the  associated  three-dimensional  representation.  These  grid 
areas  provide  the  user  with  some  frame  of  reference  as  to  where  actual  surveys  were 
performed  in  relation  to  the  test  site  as  a  whole.  The  user  can  easily  toggle  back  and  forth 
between  two  and  three  dimensional  site  displays. 


Figure  13.  Two  Dimensional  Site  Representation 


Figure  14.  Three  Dimensional  Site  Representation 
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Additional  functions  can  be  performed  using  ODES  A  such  as  overlaying  a  contour  plot  of  the 
survey  site  (Figure  15);  rotating,  shading  and  rendering  surfaces;  translating;  and  zooming  the 
display  in  and  out. 


Figure  15.  Contour  Overlay  for  ODES  A  Site  Display 
4.4  Database  Management: 

ODESA  uses  the  capabilities  of  an  object-oriented  database  management  system  known  as 
ONTOS.  ONTOS  supports  multiple  clients  distributed  across  a  network.  ODESA  uses  the 
ONTOS  database  management  system  feature  to  store  and  retrieve  data.  ONTOS  provides 
ODESA  with  a  mechanism  for  reporting  target  information  on  the  screen,  revealing  information 
about  demonstrators  or  retrieving  any  of  the  information  or  data  sets  ODESA  uses  for  display  and 
processing  algorithms. 

The  following  information  is  used  by  the  ODESA  database: 

Site  Definition 
Target  Reports  and  Targets 
Detection  Rig  and  Sensor 
Performance  Weights 
Genetic  Rules 
Heuristic  Rules 
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1.  Site  Information:  Site  information  includes:  site  name,  boudary  information 
and  usage  history.  It  also  includes  the  names  of  files  that  contain  elevation, 
magnetic  and  dielectric  constant  data. 

2.  Target  Reports  and  Targets:  Target  reports  are  a  form  of  standard  data  sets 
used  by  the  JPG  Phase  I  data  analysis  team.  Target  report  information 
includes  data  submitted  by  JPG  Phase  I  demonstrators  in  ARS  format.  A 
target  report  lists  the  company  name,  date,  report  number,  environmental 
conditions,  and  target  file  name.  The  target  file  is  an  ASCII  text 
representation  of  an  Excel  spread  sheet. 

3.  Detection  Rig  and  Sensor:  Detection  rig  and  sensor  data  provide  information 
on  the  physical  configuration  of  the  detection  systems  used.  The  Detection 
Rig  Data  Set  includes  navigation,  number  of  sensors,  detection  technology, 
data  processing  type  and  towing  mechanism  information.  The  Sensor  Data 
Set  describes  the  sensor  technology  and  capabilities. 

4.  Performance  Weights:  Performance  Weights  are  derived  and  used  by 
ODESA  to  characterize  the  performance  of  a  detection  system  in  correctly 
identifying  the  existence  of  buried  objects.  The  weights  describe  the 
performance  of  a  detection  system  in  several  categories  corresponding  to 
depth,  class,  type  and  size  of  UXO.  These  weights  are  used  by  the  Analysis 
component  to  analyze  and  fuse  results.  Weights  are  derived  automatically  by 
Target  Reports  analysis,  but  can  be  input  into  ODESA  as  well. 

5.  Genetic  Rules:  Genetic  Rules  are  a  means  of  processing  information  using 
a  "dominant  trait"  survival  approach,  much  like  the  way  in  which  dominant 
characteristics  are  passed  on  from  generation  to  generation  in  humans  and 
recessive  traits  are  not.  Genetic  rules  were  written  for  four  demonstrator  data 
sets  that  were  used  for  ODESA  testing  and  verification.  These  rules  cannot 
be  applied  to  user  specified  data  sets,but  they  serve  to  demonstrate  the 
viability  of  using  this  approach.  Further  development  of  genetic  algorithms 
is  planned  for  Phase  III. 

6.  Heuristic  Rules:  Heuristic  Rules  fuse  demonstrator  target  data  sets  into  a 
final  target  data  set  that  more  accurately  describes  the  location  of  a  buried 
object.  This  fusion  process  is  accomplished  by  combining  performance 
measures,  a  likelihood  calculus,  probabilities  and  a  merging  algorithm. 
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4.5  ODESA  Analysis  Algorithms 


The  knowledge  base  for  ODESA's  data  fusion  modules  is  derived  from  processed  data 
collected  from  20  demonstrators  who  participated  at  the  JPG  Phase  I  demonstrations  held  at 
the  40-acre  test  site.  To  assist  in  the  design  of  ODESA  reasoning  components,  half  of  the 
answer  key  for  the  emplaced  objects  was  supplied  to  generate  the  rule-base.  The  target  key 
information  supplied  to  developers  consisted  of  half  of  the  number  of  targets  emplaced  in  an 
area  approximately  half  the  size  of  the  survey  site.  The  answer  key  provided  a  "training 
window"  in  which  it  was  possible  to  develop  and  test  concepts  for  the  fusion  components  of 
ODESA.  As  an  initial  step,  ODESA  developers  had  to  determine  which  aspects  of  the 
demonstrator's  detection  system  affected  performance.  Developers  had  to  answer  questions 
such  as  which  sensor  technology  could  detect  deeper  targets  and  which  systems  utilized  GPS 
or  more  accurate  navigation  systems.  ODESA  analysis  algorithms  include  a  scoring  utility, 
heurisitic  probabilistic  processing,  and  genetic  algorithm  techniques. 


4.5.1  Scoring  Utility: 

To  provide  insight  into  system  and  sensor  performance,  and  to  have  a  means  of 
gauging  data  fusion  algorithm  effectiveness,  ODESA  developers  generated  a  scoring 
utility  that  analyzes  the  target  data  against  the  baseline  data  in  the  training  area.  The 
scoring  utility  counts  the  number  of  targets  detected,  the  number  of  targets  missed 
and  the  number  of  false  reports  inside  the  training  area.  It  summarizes  performance 
based  on  categories  such  as  target  type,  target  class,  target  size,  target  depth  and 
target  material.  The  scoring  utility  provides  information  on  ordnance  type  detected 
and  accuracy  of  the  demonstrators  target  reports  .  Developers  discovered  that  many 
of  the  systems  that  did  poorly  at  identifying  targets,  provided  useful  information  in 
attempting  to  classify  those  same  objects.  It  was  determined  that  the  classification 
categories  reported  by  demonstrators  often  correlated  with  detection  performance. 
The  scoring  utility  generated  output  reports  similar  to  the  table  that  appears  in  Figure 
16. 
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54 
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47 

13 
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54 

21 
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-1 

0 

16 

12 
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64 

43 

0.67 

-1 

0 

25 

14 

0.56 

21 

9 

0.43 

-1 

0 

12 
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15 

7 
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0 
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0 

0 

9 

9 

1 

-1 

13 
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Small 

54 

18 

0.33 
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0.84 

-1 

8 

-1 

Medium 

24 

15 

0.63 

20 

12 

0.60 

-1 

3 

-1 
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30 

21 

0.70 

21 

8 

0.38 

-1 

2 

-1 

Ferric 

86 

52 

0.60 
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0.78 

-1 

0 

-1 

Non  Ferric 

1 

1 

1 

0 

0 

-1 

-1 

0 

-1 

Non  Metal 

22 

1 
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0 

0 

-1 

-1 

0 

-1 
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8 

5 

0.63 

17 

5 

0.29 

-1 

0 

-1 
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27 

14 

0.52 

0 

0 

-1 

-1 

0 

-1 
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28 
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0.72 

-1 

0 

-1 
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21 

0 

0 

0 

0 

-1 

-1 

0 

-1 
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4 
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1 

1 

1 

-1 

0 

-1 
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21 

14 

0.67 

0 

0 

-1 

-1 

0 

-1 

Single 

84 

37 

0.44 
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106 

0.67 

-1 

0 

Multiple 

4 

3 

0.75 

1 

1 

1 

-1 

0 

Non  Ordnance 

21 

14 

0.67 

79 

78 

0.99 

-1 

0 

Figure  16.  Summary  Data  for  Sample  Target  Report  Score 


The  target  score  sheets  summarized  the  correlation  of  target  data  sets  with  baseline 
data  set  information  for  detection  and  reporting  accuracy.  The  "-1 "  value  shown  in 
the  figure  represents  that  no  data  was  collected  in  this  category.  The  ODESA  scoring 
methodology  does  not  penalize  target  data  sets  that  make  multiple  reports  of  the  same 
object  with  slightly  varying  location  data.  A  possible  disadvantage  to  this  approach 
may  be  that  a  large  number  of  reports  over  a  small  area  may  or  may  not  be  due  to  the 
same  real  target,  but  the  scoring  algorithms  scores  them  as  correct  if  they  are  within 
the  search  radius.  This  same  scoring  utility  is  then  used  to  interpret  if  the  data 
analysis  performed  by  the  ODESA  advanced  processing  (heuristic  and  genetic 
algorithm  approaches)  yielded  better  results. 


4.5.2  ODESA  Heuristic  Probability  Module  fHPMi 

ODESA  utilizes  heuristic  probabilistic  processing  in  an  analysis  step  known  as  the 
HPM.  HPM  theoretically  fuses  any  number  of  demonstrator  target  data  sets  into  an 
analysis  target  data  set  that  more  accurately  describes  the  location  of  buried  objects. 
The  current  HPM  fuses  data  by  combining  performance  measures,  a  likelihood 
calculus,  probability  theory  and  a  merging  algorithm. 

The  ODESA  1.0  prototype  was  developed  with  a  closed  HPM  architecture  simply  for 
the  purposes  of  proving  that  HPM  processing  is  valuable  in  fusing  sensor  information 
from  a  variety  of  demonstrators.  Plans  to  make  the  architecture  more  open  in  order 
to  allow  the  user  to  select  data  sets  for  input  into  the  HPM  is  scheduled  for  Phase  HI. 
Future  HPM  processing  could  provide  answers  to  depth,  class,  size,  type  and 
orientation  questions. 

The  HPM  assigns  a  detection  likelihood  to  every  detected  target  in  every  target  data 
set  to  be  fused;  in  the  future  the  user  would  be  allowed  to  select  which  sensor 
information  he  would  like  to  fuse.  Using  a  heuristic  approach,  a  detection  likelihood 
is  calculated  that  combines  reporting  accuracy  probabilities.  The  decision  to  use  a 
heuristic  method  rather  that  a  conventional  probability  method  was  made  to  allow  for 
the  development  of  non-probabilistic  rules. 

After  assigning  detection  likelihoods,  the  HPM  merges  all  demonstrator  target  reports 
within  the  specified  fusion  distance  parameter.  The  merging  algorithm  is  a 
minimum-cover  algorithm  designed  to  produce  the  minimum  number  of  clusters 
(groupings  of  several  detected  anomalies  within  the  given  input  radius)  with  a  fixed 
radius  that  can  contain  all  target  reports;  this  approach  minimizes  the  number  of 
holes  that  a  remediation  team  would  have  to  dig.  Once  the  minimum  number  of 
clusters  is  found,  data  is  merged  using  a  simple  average  of  the  positions  (x,  y,  and  z). 
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Lastly,  the  HPM  process  stores  all  merged  targets  that  surpass  the  detection 
likelihood  threshold.  This  module  creates  an  analysis  target  data  set  that  can  be 
manipulated  via  the  ODESA  GUI  and  other  tools  such  as  the  data  visualization 
mechanism  and  database  management  tools. 


4.5.3  ODESA  Genetic  Algorithm  Module  (GAMI 

The  second  fusion  mechanism  to  be  utilized  by  ODESA  is  referred  to  as  the  GAM. 
The  GAM  is  modeled  on  the  way  dominant  and  recessive  traits  in  humans  are  passed 
down  from  generation  to  generation.  The  GAM  consists  of  five  major  processing 
components;  clustering,  fact  itemization,  rule-set  generating,  optimizing,  and 
ordnance  prediction.  The  first  four  components  deal  with  the  process  of  developing 
a  set  of  rules  to  drive  the  data  fusion  mechanism  in  the  Ordnance  Prediction 
Component.  The  last  component  uses  the  developed  rule-set  to  predict  the  presence 
or  absence  of  buried  ordnance  at  a  particular  location. 


GAM  Clustering  Component 

The  GAM  Clustering  Component  compiles  individual  contractor/sensor  target 
reports  into  groups  based  upon  the  proximity  of  one  detected  anomaly  to  another; 
these  groups  are  called  clusters.  This  component  requires  an  input  search  radius  for 
processing  and  examines  each  target  report,  comparing  it  to  others  nearby,  to 
determine  if  any  of  the  neighboring  reports  lie  within  the  circle  defined  by  the  cluster 
center.  A  cluster  list  is  then  generated  for  further  GAM  processing. 


GAM  Fact  Itemizer 

The  GAM  Fact  Itemizer  compiles  a  set  of  facts  concerning  each  cluster.  These 
factors  are  simple  statements  which  summarize  the  major  attributes  of  each  cluster. 
Ideally,  each  fact  would  provide  evidence  relevant  to  the  determination  of  whether 
the  cluster  contained  any  real  ordnance.  Facts  can  include:  class  of  ordnance,  sensor 
type  reporting  detection,  size  of  ordnance,  depth  of  ordnance,  type,  etc. 


GAM  Rule-Set  Generator 

The  GAM  Rule-Set  Generator  generates  the  set  of  rules  upon  which  the  fusion 
module  can  operate.  The  rule-base  was  developed  in  a  purely  heuristic  manner. 
The  basic  approach  used  is  to  create  many  different  rule-sets  and  see  how  well  each 
rule-set  performs.  The  rule  sets  with  the  best  performance  would  be  retained  for 
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future  use  on  the  ODESA  prototype  system.  The  mechanism  used  in  the  creation  of 
a  particular  candidate  rule-set  is  a  mixture  of  deterministic  and  random  processes. 
Initially  single-condition  rules  are  generated,  and  then  multi-conditions  rules  are 
added.  The  final  rule-set  demonstrated  in  ODESA  1 .0  is  a  combination  of  47-single- 
condition  rules,  46-two  condition  rules,  7-three  condition  rules  totalling  100  rules. 


GAM  Rule-Set  Optimizer: 


The  Rule-Set  Optimizer  determines  a  set  of  weights  for  the  rules  in  the  mle-set  which 
effectively  predicts  the  presence  or  absence  of  ordnance  in  clusters.  Although  there 
is  no  guarantee  that  such  a  set  of  weights  exists,  the  Rule-Set  Optimizer  tries  to 
provide  the  best  set  of  weights.  First,  the  optimizer  creates  an  integer  array  (a  x  b), 
where  "a"  represents  the  number  of  array  columns  (the  size  of  the  population  to  be 
used  by  the  genetic  algorithm)  and  "  b"  represents  the  number  of  rules  in  the  set.  The 
optimizer  then  randomly  assigns  weights  to  the  cells  of  the  array,  ranging  from  -50 
to  -1-50.  The  optimizer  scores  each  column  against  the  entire  set  based  on  true 
negatives,  false  negatives,  false  positives  and  true  positives,  where  the  definitions  of 
these  terms  follows: 


#  True  Negative:  No  object  was  predicted  and  no  object  present. 

#  False  Negative:  No  object  waspredicted,  but  an  object  was  present. 

•  False  Positive:  An  object  was  predicted,  but  no  object  was  present. 

•  True  Positive:  An  object  was  predicted,  but  no  object  was  present. 

When  all  columns  have  been  scored,  they  are  then  ranked  in  terms  of  overall 
performance.  The  columns  with  the  most  detections,  the  fewest  misses  and  fewest 
false  reports  are  scored  the  highest.  The  results  of  all  the  columns  are  then  weighted, 
the  results  are  scored,  and  divided  into  two  groups  consisting  of  the  best  and  worst 
performers.  The  weights  of  the  best  performers  are  kept  and  are  mated  to  generate 
a  new  set  of  weights  to  score.  The  mating  of  the  best  performers  is  repeated  until  the 
optimizer  determines  that  it  is  not  feasible  to  continue;  this  is  usually  determined 
when  continuing  iterations  yield  the  same  or  worse  performance  than  previous 
attempts.  At  the  moment  this  process  is  time-consuming,  time  dependence  based  on 
the  size  of  the  matrix,  the  number  of  rules  and  the  complexity  of  the  rules. 


Ordnance  Prediction  Component: 


The  Ordnance  Prediction  Component  uses  the  rule-set  developed  by  the  four  previous 
components  (GAM  Clustering  Component,  GAM  Fact  Itemizer,  GAM  Rule-Set 
Generator,  GAM  Rule-Set  Optimizer)  to  predict  the  presence  or  absence  of  buried 
objects  at  a  particular  location.  For  each  cluster  the  component  makes  a  prediction 
by  running  the  entire  rule-set  against  that  cluster  and  suimning  the  weights  for  all  the 
rules  found  to  be  true.  This  process  determines  whether  a  target  is  present. 


5.0  EVALUATION  OF  THE  ODESA  PROTOTYPE 


The  purpose  of  this  section  of  the  report  is  to  evaluate  the  progress  of  the  ODESA  1.0 
prototype  towards  meeting  the  goals  established  for  the  prototype  and  the  goals  established 
for  finished  product.  This  section  will  reiterate  prototype  and  final  product  ODESA  goals, 
and  evaluate  ODESA  1.0  against  those  goals 


5.1  Goals  of  ODESA  -  The  Final  Product: 

The  goals  of  the  ODESA  program  were  developed  when  the  program  was  initially  started. 
The  UXO-CTP  program  required  a  mechanism  to  perform  data  fusion  processing  on  sensor 
and  environmental  data  being  acquired  from  UXO  detection  systems  like  the  Subsurface 
Ordnance  Characterization  System  (SOCS),  the  Airborne  Ground  Penetrating  Radar 
(AGPR),  and  the  Man  Portable  Ordnance  Detection  System  (ManPODS)  projects  in  a 
addition  to  a  variety  of  other  data  processing  efforts  utilizing  sensor  data  in  the  standard  data 
set  format  such  as  JPG  Phase  n  and  Live  Site  Demonstrations. 

Goals  of  the  ODESA  System  (Final  Product)  include: 

#  Provide  more  accurate  and  reliable  information  on  buried  UXO,  by 
maximizing  the  probability  of  detection  and  minimizing  the  false  alarm  rate 

#  Establish  an  open  system  architecture  that  would  allow  flexibility  in  the 
utilization  of  historical,  geographical,  geological,  sensor,  known  anomaly  and 
user  input  information 

#  Provide  a  platform  (hardware  and  software  mechanism)  that  would  fuse 
sensor,  location,  environmental  and  site  specific  information  presented  in 
STD  and  SRD  formats. 
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•  Make  intelligent  decisions  on  ordnance  characteristics  such  as  location, 
buried  depth,  classification,  orientation  and  size 

•  Provide  output  information  in  a  format  useable  by  the  remote  excavator 
(STD)  and  EOD  technicians  (maps  and  tables) 

•  Modify  historical  target  files  after  ordnance  items  have  been  remediated  to 
maintain  up-to-date  historical  files  of  surveyed  areas 

•  Provide  user  friendly  operations 


5.2  Goals  of  ODESA  1.0  Prototype 

Early  on  in  the  ODESA  Phase  II  effort,  the  goal  of  the  ODESA  system  was  to  design, 
develop  and  demonstrate  a  system  that  could  be  used  to  analyze  sensor  and  site  specific 
information.  NAVEODTECHDIV  intended  to  develop  the  "final  product"  that  need  only 
minimal  refinements  to  make  the  system  operational.  After  initial  discussions  between 
NAVEODTECHDIV  and  USAEC  personnel,  it  was  determined  that  ODESA  development 
should  be  based  upon  the  successful  completion  of  several  smaller  integral  steps.  The  initial 
effort  was  chosen  to  be  development  of  STD  data  analysis  techniques.  NAVEODTECHDIV 
and  PRC  established  the  following  goals  for  the  ODESA  1 .0  prototype,  a  subset  of  the  goals 
stated  in  Section  5.1: 

•  Develop  and  demonstrate  an  interactive  graphical  user  interface  for  ODESA 
manipulation.  This  process  allows  the  user  to  easily  perform  operations  and 
data  manipulation. 

•  Analyze  sensor  information  (using  genetic  algorithms  and  heuristic  analyses) 
from  four  different  JPG  I  demonstrators  and  provide  a  more  accurate 
accounting  of  buried  UXO  detection  and  location  than  could  be  acquired 
from  any  single  sensor. 

•  Develop  software  mechanisms  for  incorporating  future  analysis  processes. 

•  Design,  develop,  and  deliver  a  prototype  system  that  demonstrates  data 
manipulation  and  data  fusion  capabilities. 

•  Develop  a  system  with  open,  flexible  and  easily  modified  software  to 
incorporate  enhancements  planned  for  the  later  development  phases. 

•  Provide  output  target  information  in  a  format  acceptable  for  remediation 
mechanisms. 
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5.3  ODESA  Evaluation: 


This  section  evaluates  the  ODESA  1 .0  prototype  against  the  parameters  outlined  in 
Section  5.2.  Each  parameter  is  addressed  separately  and  supporting  information  is 
provided  to  reinforce  conclusions  made. 


5.3.1  Graphical  User  Interface  -  Develop  and  demonstrate  an  interactive 
graphical  user  interface  for  ODESA  manipulation.  This  process  allows  the  user  to 
easily  perform  operations  and  data  manipulation. 

The  ODESA  1.0  prototype  allows  operations  to  be  easily  performed  through  a 
windows  (Motif  or  Open  Lx)ok)  environment.  In  addition  to  the  developed  ODESA 
capabilities,  the  prototype  allows  the  operator  to  utilize  Motif,  Open  Look  and 
Solaris  capabilities  already  available  under  the  SUN  SPARC  20  environment. 

ODESA  commands  are  easily  initiated  through  a  menu  and  icon  driven  approach. 
Commands  are  accessed  through  pull-down  menus  and  a  software  users  manual 
provides  easy  to  understand  information  for  inexperienced  operators.  The  prototype 
allows  the  JPG  Phase  1 40  acre  site  to  be  plotted  and  manipulated  on  the  screen,  in 
two  and  three  dimensional  formats.  Additional  data  manipulation  capabilities 
include  overlaying  grid,  contour  and  sensor  data  information,  zooming  in  and  out 
capabilities  and  modification  of  the  color  palette  to  enhance  background  to  target 
differences.  The  survey  area  can  be  three  dimensionally  rendered  and  colors  changed 
to  allow  the  user  more  visibility  in  observing  target  detections. 


5.3.2  Sensor  Analysis  -  Analyze  sensor  information  ( using  genetic  algorithms  and 
heuristic  analyses)  from  four  different  JPG  I  demonstrators  and  provide  a  more 
accurate  accounting  of  buried  UXO  detection  and  location  than  could  be  acquired 
from  any  single  sensor. 

ODESA  developers  demonstrated  the  use  of  genetic  algorithms  and  heuristic 
probabilistic  techniques  to  analyze  demonstrator  data  sets.  This  ability  was 
demonstrated  through  data  analysis  and  system  demonstration.  Through  the  data 
analysis  portion,  ODESA  generated  a  target  data  set  based  on  four  JPG  Phase  I 
demonstrators.  The  goal  was  to  determine  if  ODESA's  processing  ability  was  better 
at  determining  the  presence  of  buried  targets  than  four  individual  sensors  systems. 
ARS  analyzed  the  ODESA  data  using  the  same  JPG  Phase  I  demonstrator  data 
processing  routines.  It  was  determined  that  ODESA  minimized  the  false  alarm  rates 
incurred  by  individual  sensors.  The  system's  GUI  and  user  friendly  operations  were 
demonstrated  to  NAVEODTECHDIV  and  AEG  personnel  at  a  system  demonstration 
in  December  1994  at  McLean,  Virginia  and  at  Indian  Head,  Maryland. 
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5.3.3  Modifiable  Software  and  Hardware  -  Develop  software  mechanisms  for 
incorporating  future  analysis  processes. 

This  evaluation  criteria  can  be  separated  into  two  categories;  user  interface 
mechanisms  and  data  analysis  processes.  ODESA  developers  have  incorporated 
several  mechanisms  to  allow  additional  capabilities  to  be  added  to  the  system.  In  the 
windows  environment,  menu  options  for  planned  enhanced  features  have  already 
been  incorporated,  even  though  the  features  themselves  have  not  been  developed. 

Since  the  prototype  system  was  designed  to  analyze  JPG  Phase  I  demonstrator  data, 
the  genetic  algorithm  uses  four  specific  demonstrator  data  sets.  ODESA  1.0  does  not 
allow  the  input  of  new  site  or  sensor  information,  the  genetic  algorithm  cannot  be 
performed  on  any  other  demonstrators  than  the  four  specified,  the  operating  system 
cannot  be  upgraded  without  destroying  the  prototype.  According  to  system 
developers,  since  the  software  architecture  and  GUI  have  mechanisms  in  place  to 
make  the  architecture  more  open,  minimal  effort  will  be  needed  to  convert  the 
demonstrator  specific  and  site  specific  environment  into  a  more  open  architecture. 

The  processing  capabilities  planned  for  ODESA  2.0,  include  utilizing  semi-raw  data 
information.  Because  of  the  complexity  and  size  of  this  effort,  no  mechanisms  have 
been  put  in  place  to  address  semi-raw  data  analysis.  This  effort  will  be  time- 
consuming  and  will  involve  state-of-the-art  data  fusion  techniques  to  address  the 
buried  UXO  problem. 


5.3.4  Prototype  Development  (Hardware  and  Software)  -  Design,  develop,  and 
deliver  a  prototype  system  that  demonstrates  data  manipulation  and  data  fusion 
capabilities 

ODESA  developers  designed  a  prototype  system  based  on  the  results  of  Phase  I 
investigations  and  with  the  approval  of  the  government.  The  prototype  was  initially 
developed  using  Phase  I  information  and  was  used  to  demonstrate  genetic  algorithm 
and  heuristic  probabilistic  analysis  methods  could  provide  more  accurate  information 
on  buried  targets.  The  system  was  delivered  to  NAVEODTECHDIV,  and  Navy 
personnel  were  trained  on  the  system. 

One  problem  with  the  hardware  configuration  as  it  exists  is  that  no  external  disk 
drive  or  CD  ROM  drive  was  purchased  for  this  system,  even  though  CDs  including 
system  software  information  were  delivered  with  the  prototype.  An  external  tape 
drive  was  purchased  because  initial  data  acquisition  studies  indicated  that  a  tape  unit 
would  be  the  most  likely  candidate  for  SOCS  data  storage. 
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Also,  there  might  be  a  problem  when  data  analysis  proceeds  to  the  SRD  processing 
point.  Originally  two  workstations  were  proposed  for  Phase  n  efforts.  It  was 
planned  that  these  two  workstations  would  be  networked  to  provide  the  processing 
ability  needed  to  manipulate  and  process  data  simultaneously.  ODESA  2.0  might 
have  to  be  "ganged"  with  another  workstation  to  provide  the  needed  speed  and 
processing  ability. 

The  software  tools  used  to  develop  ODESA  were  a  combination  of  COTS  products 
and  custom-made  algorithms  (written  in  C).  Because  of  cost,  time  and  schedule 
considerations,  the  contractor  purchased  the  development  software  and  used  this 
software  to  build  ODESA.  As  a  result,  the  government  does  not  own  the 
development  tools,  cannot  modify  any  of  the  analysis  routines  developed,  and  owns 
only  a  license  to  "run"  the  executible  files.  The  Navy  intends  to  purchase  the 
development  tools  used  under  Phase  n  as  well  as  those  tools  to  be  used  under  Phase 
in.  This  will  give  the  Government  the  ability  to  modify  analysis  processes  without 
the  aid  of  any  specific  contractor. 

Several  "glitches"  were  found  to  exist  in  the  prototype  system.  One  problem 
occurred  when  the  user  attempted  to  leave  the  ODESA  program  using  the  Motif 
environment:  the  system's  configuration  would  change  to  produce  white  letters  on 
a  white  screen.  ODESA  developers  investigated  the  problem,  determined  that  it  was 
a  Solaris-generated  problem  and  provided  new  code  to  correct  the  error.  Problems 
were  also  encountered  in  the  laboratory,  when  the  prototype  was  networked  to  other 
computers.  Systems  competing  for  memory  space  sometimes  caused  ODESA  to 
crash.  Although  NAVEODTECHDIV  has  not  been  able  to  recreate  this  problem,  the 
software  user's  manual  explains  how  to  address  this  issue  and  software  is  provided 
on  tape  to  reinstall  the  program. 


5.3.5  Output  Mechanisms  -  Provide  output  target  information  in  a  format 
acceptable  to  remediation  mechanisms. 

Because  the  data  fusion  function  is  required  by  a  variety  of  remediation  mechanisms, 
the  ODESA  prototype  must  be  able  to  provide  output  files  in  tabular  form,  STD  files 
on  disk  or  tape  and  hard  copy  printouts  for  EOD  personnel  in  the  field.  ODESA  1 .0 
allows  output  of  target  information  to  a  printer  in  a  tabular  format.  The  current 
system  does  not  have  the  printer  function  activated,  and  while  the  ODESA  program 
save  a  file  to  the  hard  disk,  the  operator  must  exit  the  ODESA  program  and  utilize 
UNIX  commands  to  save  that  file  to  other  media.  The  print  command  will  have  to 
be  made  easier  to  use,  and  the  commands  used  to  save  data  in  different  formats 
should  be  made  transparent  to  the  operator.  The  output  data  is  in  a  format  compatible 
with  ARS  processing  required  for  Phase  II  development  and  was  used  to  judge  the 
performance  of  ODESA.  Under  Phase  m,  NAVEODTECHDIV  will  require  the 
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system  to  save  the  ODESA  output  information  in  ARS  and  STD  format  to  disk/tape. 
Regarding  requirements  for  ODESA  to  send  files  to  a  printer  and  provide  hardcopy 
maps  to  users,  the  developers  believe  that  incorporation  of  these  capabilities  involves 
a  relatively  minor  effort  and  that  these  capabilities  can  be  added  to  the  prototype 
utility  functions. 


6.0  RECOMMENDATIONS 

The  following  section  provides  recommendations  for  future  enhancements  of  the  ODESA 
1.0  prototype.  These  recommendations  were  obtained  from  system  developers  and 
NAVEODTECHDIV  personnel  through  system  interaction  and  design  experience.  Both 
enhancements  to  current  system  capabilities  and  ideas  for  new  development  efforts  are 
discussed  below. 


6.1  Development  of  a  Fieldable  System 

Transitioning  the  ODESA  prototype  into  a  deployable  system  will  require  the  developer  to: 
resolve  software  problems,  fine-tune  system  performance,  enhance  existing  capabilities, 
provide  general  maintenanceand  upgrades,  and  repair  the  system.  After  extensive  evaluation 
of  the  ODESA  system  by  users,  a  prioritized  list  of  performance  enhancements  should  be 
generated  for  the  Phase  n  system.  With  USAEC  approval,  this  list  of  enhancements  will  be 
generated  after  formal  review  of  this  report,  and  Phase  III  enhancements  will  then  begin. 


6.2  Genetic  Algorithm  Module  Enhancements 

The  GAM  performed  well  during  Phase  n  demonstrations.  The  GAM  was  optimized  for  an 
initial  set  of  conditions  specific  to  the  four  JPG  Phase  I  demonstrators  and  will  be  modified 
as  the  ODESA  prototype  evolves.  Because  of  limitations  in  funding,  time  and  schedule  a 
simple  genetic  algorithm  was  developed  to  demonstrate  feasibility.  It  is  recommended  that 
for  Phase  HI  analysis  of  semi-raw  data  information,  a  more  advanced  genetic  algorithm 
package  be  used  for  the  GAM.  The  contractor  recommends  using  a  software  tool  known  as 
the  Splicer  Genetic  Algorithm  Tool.  This  software  package  was  developed  by  NASA’s 
Software  Technology  Branch  and  is  currently  available  for  use  on  government  contracts  at 
no  cost.  ODESA  developers  have  obtained  a  distribution  copy  of  this  software  and  are  ready 
to  integrate  this  package  into  the  prototype. 

The  genetic  algorithm  rule-base  provides  currently  provides  no  flexibility  in  selecting  new 
data  for  fusion  or  for  adaptively  changing  the  GAM  rule-base,  and  ODESA  has  no  capability 
to  store  the  rules  as  persistent  objects  in  its  database.  This  limitation  in  rule-base  flexibility 
can  be  overcome  by  integrating  the  rule-set  structures  into  the  ODESA  ONTOS  database. 
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The  GAM  rule  base  was  developed  and  optimized  using  the  data  reported  by  four  JPG  Phase 
I  demonstrators.  It  has  no  adaptive  ability  to  develop  new  rule-sets  in  response  to  system 
changes  incurred  by  new  demonstrators  or  sensor  devices.  It  is  recommended  that  an 
adaptive  or  learning  mechanism  be  incorporated  into  the  prototype  so  that  the  system  can 
"automatically"  adjust  to  changes  in  the  input  data  stream. 


6.3  Heuristic  Probabilistic  Module  Enhancements: 

The  HPM  can  be  modified  to  incorporate  sensor,  site,  and  target  information  obtained  during 
the  remediation  process.  The  modification  process  could  be  accomplished  through  the  use 
of  fuzzy  expert  system  rules  as  they  are  acquired.  Changes  to  the  HPM  can  then  be  made  to 
improve  system  performance  based  on  specific  site,  sensor  and  environmental  considerations. 
Several  types  of  HPM  enhancements  are  suggested: 


#  Intelligent  Clustering  Approach  -  Methods  for  combining  multiple  reports  in  the 
same  survey  area  should  be  investigated,  and  identification  and  classification 
characterstics  should  be  exploited  when  generating  fuzzy  rule-set  information. 

#  Utilization  of  Target  Report  Characteristics  -Target  report  information  should  be 
expanded  to  include  target  size  and  mass  characteristics.  This  information  can  then 
be  utilized  and  compared  to  a  recognized  ordnance  reference. 

#  Enhanced  Heuristic  Rules  -  Additional  heuristic  rules  should  be  added  to  the  rule- 
base  when  new  information  on  sensor  technology  and  performance  is  acquired. 

#  Alternative  Probabilistic  Model  -  Methods  for  determining  the  likelihood  values  used 
in  HPM  processing  should  be  investigated. 

6.4  Graphical  User  Interface  Enhancements 

Several  enhancements  to  the  exisiting  GUI  component  are  recommended  by  both  ODESA 
developers  and  ODESA  users.  These  enhancements  revolve  around  the  addition  and 
modification  of  GUI  functions  to  make  the  user's  data  manipulation  job  much  easier.  The 
recommended  enhancements  are  as  follows: 

#  Oiierv  Function:  The  user  should  have  the  ability  to  choose  and  retrieve  information 
from  any  file  in  the  database  and  to  display  that  information  in  either  a  textual  or 
graphical  format  as  required.  The  current  query  function  is  limited  and  prevents  the 
user  from  performing  very  broad  queries. 
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New  Survey  Site  feature:  The  current  system  does  not  have  the  ability  to  add  new 
site  information.  JPG  Phase  I  information  was  entered  manually  by  the  ODESA 
developers;  the  input  process  was  not  automated.  By  defining  a  standard  site 
description  format  for  the  input  file,  a  new  site  could  be  created  dynamically. 

Object  Editor:  Currently,  the  user  cannot  edit  objects  displayed  by  the  system,  create 
new  objects  or  delete  unwanted  ones.  While  it  is  necessary  to  retain  a  copy  of  the 
actual  raw  data  for  archival  purposes,  the  user  may  have  a  need  to  alter  the 
information  when  generating  new  processing  strategies.  In  a  flexible  system,  the 
object  editor  should  allow  the  user  to  perform  the  follow  tasks: 

1.  Select  which  objects  should  be  included  in  the  remediation  data  set 

2.  Incorporate  comments  with  objects  inside  the  ODESA  window 

3.  Create  new  analysis  targets  based  on  the  user's  processing  experience 

4.  Alter  the  nomenclature  used  in  the  input  data  for  incorporation  into  specific 
analysis  processes. 

5.  Include  and  specify  confidence  factors  to  predictions  made. 


Graphical  Display  of  Target  Data:  The  ODESA  display  separates  analysis  targets 
from  other  types  of  targets  by  plotting  different  colors  and  symbols  on  the  screen  to 
represent  different  targets.  The  user  should  have  the  option  to  display  special 
information  such  as  the  demonstrator  or  system  that  detected  the  target,  the  search 
radius,  and  so  on. 

Display  of  Semi-Raw  Data:  Current  ODESA  processing  does  not  accommodate 
analysis  of  semi-raw  data.  When  this  new  processing  capability  is  incorporated  into 
the  prototype,  there  will  be  a  need  to  display  the  data  in  patterns  that  can  be  analyzed. 
The  data  manipulation  capabilities  demonstrated  for  the  target  data  (2-dimensional 
and  3-dimensional  plotting,  rotating  the  survey  area,  and  slicing  vieww  will  have  to 
be  incorporated  into  the  semi-raw  data  visualization  displays. 

GAM  Control  Capability  -  The  user  should  be  able  to  modify  the  GAM  module  in 
order  to  change  analysis  parameters  such  as  search  radius,  number  of  hits  and  misses, 
false  reports,  number  of  rules  in  the  rule-set,  and  the  number  of  generations  to  run  the 
GAM. 


#  On-line  help:  The  prototype  provides  no  on-line  help  assistance  to  the  user.  If  the 
system  is  to  be  field  deployed,  it  should  have  a  on-line  help  capability. 

#  Display  Additional  Site  Data:  The  user  should  be  able  display  addition  site 
information  such  as:  soil  types,  bedrock  structures,  vegetation,  lakes,  rivers,  man¬ 
made  structures,  utilities,  site  usage,  and  history  data. 

#  Neural  Network  Interface:  The  prototype  system  does  not  currently  utilize  neural 
network  processing.  This  feature  will  be  incorporated  after  semi-raw  data  processing 
is  developed. 

#  Work  Session  Preservation:  The  current  system  does  not  permit  the  user  to  save  an 
active  work  session  for  retrieval  at  a  later  time.  To  minimize  the  number  of 
redundant  processes  performed  by  the  user,  the  system  should  allow  the  user  to  save 
the  particular  work  session  and  all  accompanying  processes  for  use  at  a  later  time. 


6.5  Extended  Data  Collection 

In  addition  to  specifying  a  standard  format  for  semi-raw  data  information  (SRD)  and  target 
information  (STD),  a  standard  format  for  a  site  definition  should  be  generated  .  The  site 
definition  should  be  easily  generated  and  interpreted.  With  standardization,  little  to  no 
manual  manipulation  of  input  data  to  ODESA. 


7.0  Future  Development  Strategy  and  Conclusions 

Future  ODESA  development  encompasses  three  general  areas:  enhancement  of  standard 
data  set  analysis,  generation  of  semi-raw  data  set  processing,  and  development  of  a  fieldable 
system.  Some  of  the  specific  development  efforts  related  to  these  three  components  can  be 
addressed  concurrently,  while  others  require  a  more  serial  approach. 


7.1  Enhancements  of  STD  Analysis 

To  maximize  the  performance  of  ODESA  1.0  and  the  Government's  investment  in  data 
fusion  techniques,  the  next  ODESA  development  emphasis  should  be  making  components 
more  flexible  and  open.  Because  ODESA  1.0  was  tested  based  on  its  ability  to  analyze 
specific  demonstrator  information  JPG  Phase  I,  the  site  and  demonstrator  information  was 
manually  entered  into  the  system.  An  open  architecture  prototype  would  allow  new  site  and 
demonstrator  information  to  be  incorporated.  Displaying  of  targets  should  be  refined; 
currently  it  is  difficult  for  ODESA  operators  to  visually  distinguish  between  targets  detected 
by  different  demonstrators.  The  output  of  ODESA  processing  should  be  made  simpler  and 


40 


icon  driven.  A  printer  should  be  incorporated  as  part  of  the  ODESA  system  to  generate  color 
output  plots  and  remediation  tables. 


7.2  SRD  Processing 

From  processing  target  information,  ODESA  developers  learned  that  manufacturer  specific 
processing  of  sensor  data  greatly  influences  the  performance  of  sensor  systems.  Companies 
that  demonstrated  at  JPG  Phase  I  and  utilized  the  same  sensor  technology  had  performance 
level  that  varied  enormously.  For  a  knowledge-based  expert  system  to  provide  more  accurate 
and  reliable  UXO  information,  more  sensor  processing  information  would  have  to  be 
incorporated  into  ODESA  processing.  NAVEODTECHDIV  intends  to  pursue  development 
of  semi-raw  data  processing.  ODESA  could  then  process  target  information  on  SRD  and 
STD  information.  SRD  analysis  is  a  perfect  application  for  developing  neural  network 
algorithms  and  fuzzy  logic  processing  routines.  ODESA  could  "learn"  from  past  experiences 
with  different  sensor  systems  and  different  survey  environments.  This  would  allow  the 
development  of  a  system  that  acquires  new  knowledge  and  insight  with  changes  in  sensor 
technology  and  environmental  conditions.  The  SRD  processing  could  be  performed  with  or 
without  STD  analyses.  The  results  of  both  STD  and  SRD  processing  mechanisms  could  be 
compared  to  determine  if  manufacturer  specific  sensor  analyses  deteriorated  the  performance 
capabilities  of  the  sensors  themselves.  This  comparison  of  analyses  can  also  determine 
whether  manufacturer  specific  algorithms  are  really  necessary  in  determining  the  location, 
orientation  and  classification  of  buried  unexploded  ordnance  or  raw  sensor  data  alone  is  all 
that  is  needed. 


7.3  Fielding  a  system 

In  order  to  field  ODESA,  several  enhancements  would  have  to  be  made  to  the  prototype 
system:  provide  a  help  capability,  incorporate  software  checks  to  insure  that  the  operator 
is  not  choosing  a  feature  or  a  function  outside  the  system's  abilities,  test  the  software  and 
debug  as  necessary,  document  input  and  output  data  in  a  specific  format,  and  develop  error 
messages. 


7.4  Concluding  Remarks 

The  information  provided  in  Sections  7.1  through  7.3  outline  the  recommended  future 
strategy  for  the  ODESA  data  fusion  project.  Enhancements  of  STD  analysis  routines  and  the 
generation  of  SRD  analysis  routines  could  be  performed  concurrently,  if  funding  and 
schedule  allow.  Fielding  an  ODESA  system  would  either  have  to  be  performed  for  STD 
processing  only,  or  for  SRD  and  STD  analysis. 
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A  great  deal  of  knowledge  was  gained  from  the  work  performed  during  Phases  I  and  n  of 
ODESA  development.  Contractor  and  government  personnel  learned  that  manufacturer 
specific  sensor  processing  is  as  much  a  part  of  sensor  system  performance  as  the 
characteristics  of  the  sensor  being  used.  Heuristic  probabilistic  and  genetic  algorithm  data 
fusion  techniques  were  proven  as  a  means  for  reducing  the  false  alarm  rates  inherent  in  the 
performance  of  single  sensor  based  systems.  Flexibility  in  software  and  hardware 
architecture  is  a  must,  and  a  large  portion  of  the  ODESA  prototype  was  developed  without 
this  flexibility.  Although  the  prototype  validated  the  use  of  data  fusion  mechanisms  to 
provide  more  accurate  and  reliable  information  on  buried  UXO,  the  input  and  output 
architecture  was  built  in  a  closed  environment.  As  a  result,  the  prototype  does  not  allow  an 
operator  to  process  information  acquired  from  any  other  test  site  or  from  any  other  sensor 
manufacturer  or  system.  NAVEODTECHDFV  concludes  that  additional  effort  should  be 
expended  to  make  the  configuration  open,  provide  semi-raw  data  analysis  and  if  these  tasks 
are  successful,  modify  the  prototype  to  a  fieldable  system. 
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